Clickbait-analysis

Research Question

How to best tell if a video is click-bait or not?

Overview

We chose this topic because we like watching YouTube videos. A big issue with finding a Youtube you enjoy is avoiding clickbait videos. When you click on one, you waste time realizing its a clickbait video and then have to search for what you want again. There are many features of a video you can look at to tell that it is a clickbait video without having to watch part of the video. In this analysis, we will identify these features.

library(dplyr)

Registered S3 method overwritten by 'dplyr':
  method           from
  print.rowwise_df     

Attaching package: 㤼㸱dplyr㤼㸲

The following objects are masked from 㤼㸱package:stats㤼㸲:

    filter, lag

The following objects are masked from 㤼㸱package:base㤼㸲:

    intersect, setdiff, setequal, union

library(ggplot2)
library(stringr)
library(tidyverse)

-- Attaching packages --------------------------------------- tidyverse 1.2.1 --
<U+2713> tibble  2.1.3     <U+2713> readr   1.3.1
<U+2713> tidyr   1.0.0     <U+2713> purrr   0.3.2
<U+2713> tibble  2.1.3     <U+2713> forcats 0.4.0
-- Conflicts ------------------------------------------ tidyverse_conflicts() --
x dplyr::filter() masks stats::filter()
x dplyr::lag()    masks stats::lag()

library(stringi)

library(qdap)

Loading required package: qdapDictionaries
Loading required package: qdapRegex

Attaching package: 㤼㸱qdapRegex㤼㸲

The following object is masked from 㤼㸱package:ggplot2㤼㸲:

    %+%

The following object is masked from 㤼㸱package:dplyr㤼㸲:

    explain

Loading required package: qdapTools
Registered S3 method overwritten by 'data.table':
  method           from
  print.data.table     

Attaching package: 㤼㸱qdapTools㤼㸲

The following object is masked from 㤼㸱package:dplyr㤼㸲:

    id

Loading required package: RColorBrewer
Registered S3 methods overwritten by 'qdap':
  method               from
  t.DocumentTermMatrix tm  
  t.TermDocumentMatrix tm  

Attaching package: 㤼㸱qdap㤼㸲

The following object is masked from 㤼㸱package:forcats㤼㸲:

    %>%

The following object is masked from 㤼㸱package:purrr㤼㸲:

    %>%

The following object is masked from 㤼㸱package:tidyr㤼㸲:

    %>%

The following object is masked from 㤼㸱package:stringr㤼㸲:

    %>%

The following object is masked from 㤼㸱package:dplyr㤼㸲:

    %>%

The following object is masked from 㤼㸱package:base㤼㸲:

    Filter

library(SentimentAnalysis)

Attaching package: 㤼㸱SentimentAnalysis㤼㸲

The following object is masked from 㤼㸱package:base㤼㸲:

    write

Imports and starting analysis

Here we load and get our data ready for analysis. The first dataset is from scraping YouTube videos and taking all features off it. We took title, likes, dislikes, views, description, comments, comment count, and if its a clickbait video or not. The second data set is a kaggle data set with many more titles so we can analyize the specific titles of videos better. The reasoning behind wanting more titles is because that is the easiest feature to look at to tell if a video is clickbait so we wanted to look at that further.

ClickbaitData <- read.csv("dataset.csv") #import 1
ClickbaitData2 <- read.csv("clickBait_Data.csv") #import 2
ClickbaitData2 <- ClickbaitData2 %>%
  mutate(clickbait = recode(clickbait, `0` = "False", `1` = "True")) %>%
  rename(class = clickbait) %>%
  rename(title = titles)
JoinedData <- merge(x = ClickbaitData, y = ClickbaitData2, by = "title", all = TRUE)
JoinedData %>%
  head()

Analysis 2: Trying spreading the data by classes.

SpreadTest <- ClickbaitData %>% 
  spread(key=class, value=dislikeCount)
head(SpreadTest)

# unstack narrow to wide

We start to see that dislikeCount has some kind of impact when we observed that there are higher number of dislikes on clickbait videos.

Analysis 3: Exploring the dataset by filtering the classes

Here we are analyzing the data more and looking into what videos are clickbait and not.

# analyzing data
IsClickbait <- ClickbaitData %>%
  filter(class != "False")
NotClickbait <- ClickbaitData %>%
  filter(class == "False")
head(NotClickbait)

Analysys 4: Number of capital letters

We see if the number of capital tletters in titles show any significance as a factor or not.

LenCaps <- ClickbaitData %>%
  mutate(capsLen = stri_count_regex(title, "[A-Z]"))
LenCaps %>%
  ggplot(aes(x = class, y = capsLen)) + geom_bar(stat = "identity")

We see there’s a strong correlation between the average number of uppercase letters in a clickbait video to a video that is not clickbait.

Analysis 4: Analysing how length of titles are related to class

Looking at the plot below, we can see that there is a normal range of title lengths from 10-100 characters and then if a video is clickbait, it can be less that the 10 character average.

 
LenData <- ClickbaitData %>%
  mutate(titleLen = stri_length(title))
LenData <- ClickbaitData %>%
  mutate(titleLen = stri_length(title))
LenData %>%
  ggplot(aes(x = class, y = titleLen)) + geom_point()

NA
NA

Analysis 5: Analysing dislikeCount as a factor

Here we see if a video has more dislikes, it is going to have a much higher chance to be a clickbait video.

LenData %>%
  head(2000) %>%
  group_by(class) %>%
  summarise(dislikeCount = mean(dislikeCount))%>%
  ggplot(aes(x = class, y = dislikeCount)) + geom_bar(stat = "identity")

The above graph made a lot of sense. We do not like clickbait videos so many people would dislike it. Now on clickbate videos, uploaders can give it fake likes so next we wanted to see if there was a relationship to the ratio of likes to dislikes. We see that no matter the likes, there will still be a high amount of dislikes telling us that a video is clickbait. ### Analysis 6: Analysing the ratio of dislikes to likes as a factor Now, we check for the LikeCounts vs DislikeCounts and analyze if their ratio is making a difference.

LenData %>%
  head(1000) %>%
  ggplot(aes(x = likeCount, y = dislikeCount, color = class)) + geom_point()

It looks like a video being a clickbait has a higher dislike to likes ratio and also has many outliers.

Analysis 7: Sentimental Analysis

Up next for analysis is sentimental analysis. Using the Qdap’s library function to get polarities of a string, we see if clickbait titles have more polarities - meaning being more positive, happy, which would attract people to click on these videos.

PolData<- LenData %>%
  mutate(polarity_title = counts(polarity(title))[, "polarity"])

  Some rows contain double punctuation.  Suggested use of `sentSplit` function.

PolData%>%
  group_by(class)%>%
  summarise(meanPol = mean(polarity_title))%>%
  ggplot(aes(x = class, y = meanPol)) + geom_bar(stat = "identity")

We see there’s a huge difference in polarities and a clickbait video is more likely to have higher levels of polarities. ### Decision Tree Since we found that polarities and the ratio of likes to dislikes are the strongest factors in deciding whether a video is clickbait or not, we decided to see what a decision tree would look like considering both of these factors.

#Decision Tree
PolData<- PolData%>%
  mutate(class = recode(class, `True` = 1, `False` = 0))

mod1 <- party::ctree(
  class ~ polarity_title + likeCount/dislikeCount,
  data=PolData) 
plot(mod1, type="simple")

Conclusion

To conclude, we had a huge dataset and a lot of features that could help us analyse if a video is clickbait or not. Our key visualizations and analysis found that a title’s polarity, dislikeCount to like Count ratio are 2 of the main factors which can help us know if a video is clickbait or not. We could use this analysis further to create Machine Learning algorithms which can tlell if a video is clickbait or not in real time.